1. Field of the Invention
The present invention relates to testing computer hardware and software and more particularly to a method, apparatus and program storage device for deploying and testing computer hardware and software in heterogeneous multipath computer systems.
2. Description of Related Art
Many enterprise computer products require testing in a systems environment with many components combined together via networks. For example, an enterprise level storage server solution typically includes the storage server as well as storage management console control software designed to run on one or more workstations coupled to the storage server via a network. Part of the process of developing a new storage server includes testing the storage server at various stages of development in a network coupled to storage management consoles via a network.
A conventional approach to setting up testing of a computer hardware and or software product under development involves having a development team manually set up a physical resource, such as a server, in a desired manner and often install one or more software packages as part of the test environment. The Rational® Tool Set is an example of a conventional tool set for deploying tests on a given platform, such as for testing a Linux server. Such conventional tool sets are designed for use in testing finished products where each unit of the product being tested should have essentially the same functionality, therefore any test failure indicates a product malfunction. These conventional tool sets are not designed to handle test deployment in an unstable networked systems environment. Therefore in a product development environment where it is necessary to test a range of product build releases at varying stages of development and functionality a test team has to manually rebuild the test suites to test each different stage of build release. Typically some portion of the development team has to drop their development work to manually test the build release to ensure that the build release to be delivered to a test team is in a usable state that will allow for meaningful testing and/or interaction with external interfaces to occur. A common procedure today is to run a standard test suite (commonly called a smoke test) against the build release to ensure that it meets a certain minimum set of criteria before being deployed for more comprehensive testing.
There are clear drawbacks with this conventional approach to testing products under development including that it takes a great deal of time to manually perform many aspects of the process of evaluating a product build to determine whether it is ready for more comprehensive testing. For example, conventional approaches to test deployment are often plagued by functional limitations in the command line interface (“CLI”), graphical user interface (“GUI”), or application program interface (“API”). Furthermore, since the process is manual there exists a large degree of user influence that dictates what the outcome of the evaluation will be, for example, certain test cases might be skipped. Furthermore, as with any human-executed task, a test case result may be inaccurate based on configuration errors, missed steps or plain and simple misinterpretation and/or misunderstanding of the resulting output. This can turn out to be not only time consuming but quite costly if a software or hardware error escapes into the field with the product.
It can be seen that there is a need for an improved product test deployment and execution process to improve the efficiency, accuracy and reliability of the process.